RE unittest_windows_cpu_py3.10 and unittest_windows_gpu_py3.10

The error happens when it tries to install sentencepiece. Seemingly there is no binary distribution of sentencepiece on Windows for Python 3.10, so it falls back to source installation and fails.

Googling the issue, I do not see an error reported, but I found this google/sentencepiece#591 which I assume @seemethere hit the same issue when adding 3.9 support. So we need to ask sentencepiece maintainer to publish 3.10 binary including Windows.

Since sentencepiece is only an optional dependency, I think we can skip the tests and avoid installing sentencepiece. We will need to add some workaround to skip the tests that require sentencepiece.

Once the installation of sentencepiece is conditioned on Python version and unit test starts running, then we can merge this PR and we can take over to follow-up on skipping part.

RE unittest_linux_cpu_py3.10 unittest_macos_cpu_py3.10

This is an unexpected failure. I suspect there might be a regression about TorchScript on PyTorch core, as it is working on other versions.
We can also look into it to see if changing the call site would be a valid workaround. So ultimately you can ignore the failure in this PR.

I suspect that the same failure will happen on Windows.

closed and reopened to regenerate the tests

To disable sentencepiece we need to add a check here for windows:

To disable sentencepiece we need to add a check here for windows:

audio/.circleci/unittest/linux/scripts/install.sh

Lines 60 to 64 in bbdbd58

	if [[ "$(python --version)" = *3.9* ]]; then
	# Numba isn't available for Python 3.9 except on the numba dev channel and building from source fails
	# See https://github.com/librosa/librosa/issues/1270#issuecomment-759065048
	NUMBA_DEV_CHANNEL="-c numba/label/dev"
	fi

@seemethere Just in case you have not heard, sentencepiece is an optional dependency, so ultimately you can ignore the test failures related to sentencepiece, as long as it builds fine.

Although the test failure related to TorchScript on Tacotron2 seems like a real issue of PyTorch core on Python 3.10. I am planing to file a issue once this PR is merged.

Although the test failure related to TorchScript on Tacotron2 seems like a real issue of PyTorch core on Python 3.10. I am planing to file a issue once this PR is merged.

Should we just disable these tests for Python 3.10 in the interim?

So I found the reason why we need conda-forge for cudatoolkit=11.5. Basically cudatoolkit=11.5 from either nvidia or conda-forge has the following requirements:

cudatoolkit 11.5.1 hcf5317a_9
-----------------------------
file name   : cudatoolkit-11.5.1-hcf5317a_9.tar.bz2
name        : cudatoolkit
version     : 11.5.1
build       : hcf5317a_9
build number: 9
size        : 798.4 MB
license     : LicenseRef-NVIDIA-End-User-License-Agreement
subdir      : linux-64
url         : https://conda.anaconda.org/conda-forge/linux-64/cudatoolkit-11.5.1-hcf5317a_9.tar.bz2
md5         : cba8f073087596d11b3c96064cd3b90c
timestamp   : 2022-01-11 11:13:26 UTC
constraints : 
  - __cuda >=11.5.1
dependencies: 
  - __glibc >=2.17,<3.0.a0
  - libgcc-ng >=9.4.0
  - libstdcxx-ng >=9.4.0

The libgcc-ng >=9.4.0 seems innocuous at first but the highest available version from the default anaconda channel is actually libgcc-ng = 9.3.0.

This means that in order to actually install cudatoolkit=11.5 from any channel you will need to also include -c conda-forge since that is the only channel that includes libgcc-ng >=9.4.0, which is very unfortunate.

I'm including a commit here to enable conda-forge just for CUDA 11.5 builds but we'll also need to make a note of this in our installation matrix as well

@seemethere has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@seemethere has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

Hey @seemethere.
You merged this PR, but labels were not properly added. Please add a primary and secondary label (See https://github.com/pytorch/audio/blob/main/.github/process_commit.py)

Is the use of Python 3.10 still in development? When running torchaudio in my workflows, everything works fine for Windows and Ubuntu running 3.10, but with MacOS I get the following error. Thanks.

File "/Users/runner/.local/share/virtualenvs/drum-shape-predictor-Lsu0jyfE/lib/python3.10/site-packages/torchaudio/__init__.py", line 1, in <module>
    from torchaudio import _extension  # noqa: F401
File "/Users/runner/.local/share/virtualenvs/drum-shape-predictor-Lsu0jyfE/lib/python3.10/site-packages/torchaudio/_extension.py", line 67, in <module>
    _init_extension()
File "/Users/runner/.local/share/virtualenvs/drum-shape-predictor-Lsu0jyfE/lib/python3.10/site-packages/torchaudio/_extension.py", line 61, in _init_extension
    _load_lib("libtorchaudio")
File "/Users/runner/.local/share/virtualenvs/drum-shape-predictor-Lsu0jyfE/lib/python3.10/site-packages/torchaudio/_extension.py", line 51, in _load_lib
    torch.ops.load_library(path)
File "/Users/runner/.local/share/virtualenvs/drum-shape-predictor-Lsu0jyfE/lib/python3.10/site-packages/torch/_ops.py", line 2[20](https://github.com/lewiswolf/drum-shape-predictor/runs/6352853628?check_suite_focus=true#step:11:20), in load_library
    ctypes.CDLL(path)
File "/Users/runner/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/ctypes/__init__.py", line 374, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: dlopen(/Users/runner/.local/share/virtualenvs/drum-shape-predictor-Lsu0jyfE/lib/python3.10/site-packages/torchaudio/lib/libtorchaudio.so, 6): Symbol not found: __ZN2at8internal15invoke_parallelExxxRKNSt3__18functionIFvxxEEE
Referenced from: /Users/runner/.local/share/virtualenvs/drum-shape-predictor-Lsu0jyfE/lib/python3.10/site-packages/torchaudio/lib/libtorchaudio.so
Expected in: /Users/runner/.local/share/virtualenvs/drum-shape-predictor-Lsu0jyfE/lib/python3.10/site-packages/torch/lib/libtorch_cpu.dylib
 in /Users/runner/.local/share/virtualenvs/drum-shape-predictor-Lsu0jyfE/lib/python3.10/site-packages/torchaudio/lib/libtorchaudio.so
Error: Process completed with exit code 1.